Hybrid Syntactic Category Induction

Much research has been devoted to the task of learning lexical classes from unannotated input text. Among the chief difficulties facing any approach to the unsupervised induction of lexical classes are that of token-level ambiguity and the classification of rare and unknown words. Following the work of previous authors, the initial stage of syntactic category induction is treated in the current approach as a clustering problem over a small number of highly frequent word types. An iterative procedure making use of Zipf’s law to generate the clustering schedule classifies less frequent words based on the monotonic Bernoulli entropy of expected co-occurrence probability with respect to the clusters output by the previous stage, employing a fuzzy cluster membership heuristic to approximate type-level ambiguity and reduce error propagation in a simulated melting procedure. In a second processing phase, cluster membership probabilities output by the final clustering stage are used in a procedure for the recovery of context-dependent token-level ambiguity resolution. The induced classifications are evaluated with a meta-modelling strategy intended to capture their expected linguistic utility.